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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address -- 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )K Responsive to communication(s) filed on 06 April 2004 . 
2a)KI This action is FINAL. 2b)D This action is non-final. 

3) Q Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

dosed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) KI Claim(s) 1-3.5-19.21 and 23 is/are pending in the application. 

4a) Of the above claim(s) ; is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) KI Claim(s) 1-3,5-19.21 and 23 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) 13 The drawing(s) filed on 06 June 2000 is/are: a)KI accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1 .121 (d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 1 19 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 1 19(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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DETAILED ACTION 

1. This action is responsive to communications: amendment filed 4/6/2004, IDS filed on 
5/11/2004. 

2. The objection to the claims because the lines are crowded too closely together has been 
dropped as a result of the new listing of claims including lines spaced one and one-half spaces 
apart on good quality paper. 

3. The Examiner apologizes for the oversight of the reference to the rejection of claim 22 in 
the previous Office Action mailed 12/29/2003. The rejection was detailed on page 7, but 
accidentally omitted by the Examiner in the statement of rejection on page 2. 

4. Claims 1-3, 5-19, 21, and 23 are pending in the case. Claims 1, 10, 19, and 23 are 
independent claims. 

Claim Rejections - 35 USC § 103 

5. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

6. Claims 1-3, 5-19, 21, and 23 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Beauregard et al. (hereafter referred to as Beauregard), US 5,974,413 
filed 07/03/1997 in view of Bays et al. (hereafter referred to as Bays), US 6,519,603 filed 
10/28/1999. 
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Regarding independent claim 1 and dependent claim 5, Beauregard teaches receiving 
a string of text in a recognizer library in fig. 7 and col. 5 lines 12-56. Beauregard does not teach 
using recognizer plug-ins. Beauregard also does not specifically teach annotating a string of text 
with a plurality of labels and transmitting the labels to the application program module. Bays 
teaches annotating a string of text with a plurality of labels in fig. 1, 4, col. 2 lines 47-50, and col. 
3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable for one of ordinary skill to use plug-in software modules 
for the recognizer library of Beauregard. Plug-in software was well known to one of ordinary 
skill at the time of the invention and would have allowed for simple modification of and 
enhancement of the text strings which the recognizer library could have discerned. It would have 
been obvious and desirable to have annotated the text string with one of the service scripts of 
Beauregard to enhance the semantic interpretation (Bays col. 2 line 48) of the text string. 

Beauregard teaches comparing a string of text with a plurality of stored strings to 
determine a match in fig. 7 and col. 5 lines 12-56. Beauregard teaches performing an action 
based on a match in fig. 7 and col. 5 lines 12-56. Beauregard does not teach labeling the string 
of text with an associated stored label of the matched stored string. Bays teaches attaching a 
semantic label to a string of text in fig. 1 and 4, col. 2 lines 47-50, and col. 3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable to have modified the action of Beauregard to have 
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attached a semantic label to the string of text so that the semantic interpretation of the string of 
text could have been known. 

Regarding dependent claim 2, Beauregard does not teach synchronizing a plurality of 
labels received from the recognizer plug-in. It would have been obvious to one of ordinary skill 
in the art at the time of the invention to have designed the software to have synchronized the 
labels received before transmitting them to the application program module so that the program 
module could have processed the received labels in an organized way. 

Regarding dependent claim 3, Beauregard teaches receiving a plurality of labels in an 
action library in fig. 7 and col. 5 lines 12-56. Beauregard teaches displaying a menu displaying a 
plurality of actions based on a label in fig. 9. Beauregard does not teach using action plug-in 
software. It would have been obvious and desirable for one of ordinary skill to use plug-in 
software modules for the action library of Beauregard. Plug-in software was well known to one 
of ordinary skill at the time of the invention and would have allowed for simple modification of 
and enhancement of the actions implemented by the action library. 

Regarding dependent claim 6, Beauregard teaches performing a text string service in 
fig. 7 and col. 5 lines 12-56 which would have modified the electronic document being worked 
on. 

Regarding dependent claim 7, Beauregard teaches causing the application program 
module to fire an event within an object model of the application program module and causing a 
piece of code associated with the event to be executed when at least one of the plurality of labels 
is determined in fig. 7, 9, and col. 5 lines 12-56. 
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Regarding dependent claim 8, Beauregard does not teach determining the language of 
the string of text if the language is not recognized by the recognizer library. Language software 
was well known to one of ordinary skill in the art at the time of the invention. It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to have 
combined language software with Beauregard to created the claimed invention. It would have 
been obvious and desirable to have determined the language of unrecognized commands so that 
invention could have been geographically portable and would have been able to have been 
marketed in countries using a different language than the default of the invention. 

Regarding dependent claim 9, Beauregard teaches the use of multiword strings to 
determine actions in fig. 7 and col. 5 lines 12-56. It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have used this ability to implement context 
aided annotation. Use of context to implement annotation was well known in the art at the time 
of the invention and it would have been obvious and desirable to have used it so that the 
annotations of the text strings could have been more accurate. 

Regarding independent claim 10, Beauregard teaches determining whether an entered 
string of text matches one of a plurality of stored strings and determining an action if the string is 
matched in fig. 7 and col. 5 lines 12-56. Beauregard does not teach determining a label 
associated with the matched stored string. Bays teaches attaching a semantic label to a string of 
text in fig. 1 and 4, col. 2 liens 47-50, and col. 3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable to have modified the action of Beauregard to have 
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attached a semantic label to the string of text so that the semantic interpretation of the string of 
text could have been known. 

Regarding dependent claim 11, Beauregard teaches determining a set of actions 
associated with a string of text in fig. 7 and 9, and col. 5 lines 12-56. Beauregard does not teach 
a label associated with the string of text. Bays teaches attaching a semantic label to a string of 
text in fig. 1 and 4, col. 2 lines 47-50, and col. 3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable to have modified the action of Beauregard to have 
attached a semantic label to the string of text so that the semantic interpretation of the string of 
text could have been known. 

Regarding dependent claim 12, Beauregard teaches in fig. 9 and col. 5 lines 12-56 
displaying an indication indicating that a label has been found. 

Regarding dependent claim 13, Beauregard teaches determining that a user has selected 
a string of text and in response, displaying a plurality of actions to the user in fig. 7 and 9, and 
col. 5 lines 12-56. 

Regarding dependent claims 14 and 15, Beauregard teaches receiving an indication that 
one of the plurality of actions has been selected and in response to receiving an indication that 
one of the plurality of actions has been selected, then causing the application program module to 
execute the selected action in fig. 7 and 9, and col. 5 lines 12-56. 

Regarding dependent claim 16, Beauregard teaches wherein the application program 
module executes the selected action by determining whether an action library assigned to the 
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action is available and if so, then receiving instructions from the action dynamic link library 
assigned to the selected action in fig. 7 and 9, and col. 5 lines 12-56. 

Regarding dependent claim 17, Beauregard does not teach if an action plug-in dynamic 
link library is not available, then using a Uniform Resource Locator assigned to the action to 
navigate to a Web site and download the action plug-in dynamic link library. Beauregard does 
teach the use of third party software in fig. 7. The technique of using a Uniform Resource 
Locator assigned to the action to navigate to a Web site and download software in the event it 
was not available was well known to one of ordinary skill in the art at the time of the invention 
and would have been obvious to have included in Beauregard so that it could have automatically 
stayed up-to-date. 

Regarding dependent claim 18, Beauregard does not determine metadata associated 
with the string of text. Bays teaches determining metadata associated with a string of text in fig. 
1 and 4, col. 2 lines 47-50, and col. 3 lines 53-56. It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have combined Bays into Beauregard to 
have created the claimed invention. It would have been obvious and desirable to have used the 
metadata identification of Bays to have improved the action determination of Beauregard. 

Regarding independent claim 19, Beauregard teaches an application program module 
for creating an electronic document in col. 5 lines 51-56. Beauregard teaches a recognizer 
library and an action library in fig. 7 and col. 5 lines 12-56. It would have been obvious to one 
of ordinary skill in the art at the time the invention was made to have connected the recognizer 
and action libraries of Beauregard to an application program module for creating electronic 
documents so that documents could have been produced.more efficiently. 
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Beauregard teaches the use of third party software in fig. 7, but does not specifically 
teach the use of plug-ins. Plug-ins were well known at the time of the invention for allowing 
easy modification and enhancement of software. It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have combined the knowledge of plug-ins 
with the recognizer library so that the library could have been easily modified and enhanced. 

Beauregard teaches determining whether an entered string of text matches one of a 
plurality of stored strings and determining an action if the string is matched in fig. 7 and col. 5 
lines 12-56. Beauregard does not teach determining a label associated with the matched stored 
string. Bays teaches attaching a semantic label to a string of text in fig. 1 and 4, col. 2 lines 47- 
50, and col. 3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable to have modified the action of Beauregard to have 
attached a semantic label to the string of text so that the semantic interpretation of the string of 
text could have been known. 

Regarding dependent claim 21, Beauregard teaches the use of third party software in 
fig. 7, but does not specifically teach the use of plug-ins. Plug-ins were well known at the time 
of the invention for allowing easy modification and enhancement of software. It would have 
been obvious to one of ordinary skill in the art at the time the invention was made to have 
combined the knowledge of plug-ins with the action library so that the library could have been 
easily modified and enhanced. 
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Regarding independent claim 23, Beauregard teaches an application program module 
for creating an electronic document in col. 5 lines 51-56. It would have been obvious to one of 
ordinary skill in the art at the time the invention was made to have connected the recognizer and 
action libraries of Beauregard to an application program module for creating electronic 
documents so that documents could have been produced more efficiently. 

Beauregard teaches the use of third party software in fig. 7, but does not specifically 
teach the use of plug-ins. Plug-ins were well known at the time of the invention for allowing 
easy modification and enhancement of software. It would have been obvious to one of ordinary 
skill in the art at the time the invention was made to have combined the knowledge of plug-ins 
with the recognizer library so that the library could have been easily modified and enhanced. 

Beauregard teaches determining whether an entered string of text matches one of a 
plurality of stored strings and determining an action if the string is matched in fig. 7 and col. 5 
lines 12-56. Beauregard does not teach determining a label associated with the matched stored 
string. Bays teaches attaching a semantic label to a string of text in fig. 1 and 4, col. 2 lines 47- 
50, and col. 3 lines 53-56. 

It would have been obvious to one of ordinary skill in the art at the time the invention 
was made to have combined Bays into Beauregard to have created the claimed invention. It 
would have been obvious and desirable to have modified the action of Beauregard to have 
attached a semantic label to the string of text so that the semantic interpretation of the string of 
text could have been known. 



Response to Arguments 
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7. Applicant's arguments filed 4/6/2004 have been fully considered but they are not 
persuasive. Regarding Applicant's arguments in pages 7-10 that the combination of Beauregard 
and Bays fails to teach all of the limitations in claims 1, 10, 19 and 23, the Examiner believes the 
combination of the art is valid and does teach the limitations of the claimed invention when 
combined by one of ordinary skill in the art at the time of the invention. The Examiner's best 
interpretation of the claimed invention is a method and system for semantically labeling strings 
and providing user actions based on the types of the labels associated with the string. 

The Examiner has applied the reference of Beauregard et al. (hereafter referred to as 
Beauregard) to teach the ability of associating words or labels with specific kinds of actions. 
The Examiner believes this teaching is essentially identical to the matching of labels and 
associated actions in the claimed invention. The words of Beauregard are entered into a 
computer by a user. What Beauregard does not teach is attaching the words as an annotation, or 
label, to a string to enhance the string. As a result, the Examiner has additionally applied the 
reference of Bays et al. (hereafter referred to as Bays) to teach semantically labeling a string. 

Bays teaches annotating database material, which in the abstract Bays teaches may be 
text as well as many other forms of data. The text constitutes a string and an annotated string 
constitutes a semantically labeled string. Therefore the string is enhanced by the annotation. 
Bays teaches in the abstract that the annotations may be used at links to related material, so the 
Examiner believes it is reasonable to expect that one of ordinary skill in the art could have used 
the teaching of Beauregard to have used the annotation to have linked to one or more actions 
related to the string instead of just other database material. The Examiner believes the 
combination of Beauregard and Bays by one of ordinary skill in the art at the time of invention 
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would have resulted in semantically labeled strings, wherein the labels were empowered to have 
provided the user with actions associated with each of the labels, which the Examiner believes 
reads upon the claimed invention. 



Conclusion 

8. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 

9. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Peter J Smith whose telephone number is 703-305-593 1 . The 
examiner can normally be reached on Mondays-Fridays 7:00am-3:30pm. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Joseph H Feild can be reached on 703-305-9792. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 



PJS 

June 3, 2004 
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